

In [1]:

    
import numpy as np

Problem and intuition

We are now to find a linear map from the representation of an image to the probabilities for each classes. For CIFAR-10, the representation of an image is $32\times 32\times 3 = 3072$ real numbers, and we have $10$ classes overall. So we are to find a function:

$$H(X) = W X + b$$

where $W$ is a matrix of size $10 \times 3072$ and $b$ is a vector of size $10 \times 1$.

An advantage, compared to the kNN method, is that now we need only store $3072\times 10 + 10$ numbers instead of the whole training set, and the prediction will go must faster since we don't have to compare the input with all training cases.

The bias $b$ can be compacted into $W$ with an additional column, if we add a constant $1$ to all our input data.

We will take to model to do our linear classification, SVM and Softmax. They offer different standard to evaluate the prediction.

SVM with hinge loss

SVM



In [3]:

    
# base class for SVM and SoftMax
class LinearClassifier:
    def __init__(input_size, num_class):
        self.input_size = input_size
        self.num_class = num_class
        this.W = np.randn(num_class, input_size + 1)

    def linear_score(X):
        return W.dot(X)

    def predict(X):
        return linear_score(X)

    def loss(X, Y):
        return 0



In [4]:

    
class SVMClassifier(LinearClassifier):
    def loss(X, Y):
        Y - predict(X)



In [ ]:

    
class